International Journal for the Scholarship of 

Teaching and Learning 


1 V/W I L 

M SCHOLARSHIP of TEACHING and LEARNING 


Volume 3 | Number 1 


Article 24 


1-2009 


Operationalization of Innovative and Traditional 
Pedagogical Practices within Undergraduate 
Bioengineering Courses 


Monica F. Cox 

Purdue University - Main Campus, mfc(®purdue.edu 


Recommended Citation 

Cox, Monica F. (2009) "Operationalization of Innovative and Traditional Pedagogical Practices within Undergraduate Bioengineering 
Courses,” International Journal for the Scholarship of Teaching and Learning : Vol. 3: No. 1, Article 24. 

Available at: https://doi.org/10.20429/ijsotl.2009.030124 





Operationalization of Innovative and Traditional Pedagogical Practices 
within Undergraduate Bioengineering Courses 


Abstract 

This study operationalizes pedagogical practices using classroom observation data collected in twenty-eight 
bioengineering courses at a southeastern research university over five semesters. Using an index that 
distinguishes pedagogy reflecting principles of the “How People Learn” framework and pedagogy representing 
traditional, lecture-based instruction, the author presents five “How People Learn” instructional practices (i.e., 
guidance by the instructor, comments, praise, monitoring, and question and response) and four statistically 
significant subcategories representing traditional instructional practices (i.e., instruction by media, lecture, 
praise, and no response) that were most likely to occur within observed How People Learn-oriented classes 
and traditional, lecture-based classes, respectively. Included are details about classroom activities that 
occurred to make up the code strings representing each of these statistically significant subcategories within 
both types of courses. The operationalization of the code strings confirms the alignment of the subcategories 
with pedagogical practices that are most likely to occur within innovative 
and traditional courses. 
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Abstract 

This study operationalizes pedagogical practices using classroom observation data collected 
in twenty-eight bioengineering courses at a southeastern research university over five 
semesters. Using an index that distinguishes pedagogy reflecting principles of the "How 
People Learn" framework and pedagogy representing traditional, lecture-based instruction, 
the author presents five "How People Learn" instructional practices (i.e., guidance by the 
instructor, comments, praise, monitoring, and question and response) and four statistically 
significant subcategories representing traditional instructional practices (i.e., instruction by 
media, lecture, praise, and no response) that were most likely to occur within observed How 
People Learn-oriented classes and traditional, lecture-based classes, respectively. Included 
are details about classroom activities that occurred to make up the code strings 
representing each of these statistically significant subcategories within both types of 
courses. The operationalization of the code strings confirms the alignment of the 
subcategories with pedagogical practices that are most likely to occur within innovative 
and traditional courses. 


Introduction 

Although effective pedagogical practices have been explored independently of context, 
several researchers have noted the impact of effective pedagogical practices within 
engineering. Engineering studies are needed since engineering is a "hard/applied" field of 
study that differs from other studies given its emphasis on inquiry, its inclusion of the 
environment, and its focus on products and on the processes that are needed to produce 
such products (Biglan, 1973; Lodahl & Gordon, 1972; Neumann, Perry & Becher, 2002). 
More than in lecture-based classes, engineering classes with emphases on in-class student 
collaboration and on faculty and student interactions have been reported to increase student 
outcomes such as critical thinking skills, achievement, persistence, and attitudes (Cabrera, 
Colbeck, & Terenzini, 2001; Cooper & Robinson, 1998; Cudd & Wasser, 1999; Springer, 
Stanne, & Donovan, 1998). 

To observe engineering classroom environments, members of the assessment and 
evaluation team within the \/anderbilt University, A/orthwestern University, the University of 
Texas at Austin, and the /-/arvard/Massachusetts Institute of Technology Division of Health 
Science and Technology (VaNTH) Engineering Research Center (ERC) for Bioengineering 
Educational Technologies (VaNTH, 2007) created a four-part direct observation system 
called the VaNTH Observation System (VOS) (Harris & Cox, 2003). The primary purpose 
of the VOS was to examine the extent to which the four lenses of the "How People Learn" 
(HPL) framework, knowledge-, learner-, assessment-, and community centeredness, were 
present within observed bioengineering classrooms within the ERC (Harris & Cox, 2003). A 
knowledge-centered environment promotes learning with understanding via an organization 
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of academic knowledge around core concepts of the subject domain area and an 
understanding of the when and how to apply these concepts (Bransford, Brown, & Cocking, 
1999). Within a learner-centered environment, an instructor takes into account the 
knowledge, skills, beliefs, preconceptions, misconceptions, and learning styles of the 
students and acknowledges challenges that some students may have within a classroom. 
Assessment-centered environments allow students and faculty to make their thinking and 
learning visible and to revise this thinking through the use of feedback techniques. A 
community-centered learning environment encourages students to share norms that value 
learning and high standards. This community includes the classroom, the school, and the 
connections between the school and the larger community, including the home. 

The author developed an index called the HPL Index to parse data from the one portion of 
the VOS into code strings representing either HPL-oriented instruction or code strings 
representing traditional, lecture-based instruction. This resulted in the creation of both HPL- 
oriented instructional subcategories and traditional, or lecture based instructional 
categories. In a criterion contrast study to distinguish pedagogy in seventeen HPL-oriented 
courses and eleven traditional or lecture-based courses, the author found that the HPL- 
oriented courses, more than traditional courses, demonstrated subcategories representing 
innovation instructional pedagogical practices most aligned with the HPL framework. On the 
other hand, traditional courses, more than HPL-oriented courses, were more likely to 
demonstrate subcategories representing traditional, nonHPL-oriented instructional practices 
(Cox & Cordray, 2008). Missing within this previous study, however, were details about the 
actual classroom activities that occurred to make up the frequencies represented by each of 
these statistically significant subcategories within traditional and HPL-oriented courses. For 
this reason, the purpose of this paper is to operationalize the pedagogical practices 
associated with these occurrences. More specifically, the research questions for this study 
are (1) "What 'How People Learn'-oriented pedagogical practices are most prevalent across 
all observed engineering classrooms using the HPL Index?" and (2) "What traditional 
pedagogical practices are most prevalent across observed engineering classrooms using 
the HPL Index?" 


Methods 


Participants 

The analyses using the HPL Index are based on 182 classroom observations in twenty-eight 
(28) bioengineering courses at a southeastern research university. Such courses included 
Freshman Seminar, Systems Physiology, Biomechanics, Biotechnology, Senior Design, and 
Bioethics. Data were collected by five trained VOS observers during five semesters between 
spring 2002 and spring 2004. Classes were visited approximately eight times over the 
course of the semester, and observers collected data for the entire class period (i.e., 50 
minutes, 1 Vz hours, or two hours). Courses ranged from sophomore- to senior-level, and 
were designated as traditional or HPL by educational researchers in VaNTH prior to the 
semester that the class was observed. Seventeen of these courses primarily implemented 
HPL-oriented pedagogical practices, and eleven primarily used traditional, or lecture-based 
engineering practices. Although faculty teaching the HPL-oriented courses received no 
formal pedagogical training within their classes prior to their implementation of HPL 
materials, they consistently assisted in the design and implementation of curricula 
incorporating elements of the HPL framework within their courses. Traditional faculty, on 
the other hand, did not incorporate formally HPL-oriented materials in their courses and, 
therefore taught their courses in their usual manner. 
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Measures 

Data were collected using the HPL Index, which was confirmed to be sensitive enough to 
capture HPL-related differences in courses known to employ HPL-based or traditional 
pedagogy (Cox & Cordray, 2008). The Index analyzes data collected within the first part of 
the VOS, the Classroom Interaction Observation (CIO), which explicitly records faculty- 
student interactions, in real-time, within classes, using elements that integrate the HPL 
framework principles(Harris & Cox, 2003) (Figure 1). Each CIO coding session is 
approximately three minutes in length and is followed by three components of the VOS. As 
such, VOS observers record approximately thirty to forty-five CIO code strings at the speed 
of speech during each three minute session, resulting in hundreds of CIO code strings being 
collected per observed class session. 


WHO TO WHOM WHAT HOW MEDIA 


Professor 

Professor 

1 factual question 

Knowledge- 

Board 

Everyone 

Everyone 

2higher order question 

centered 

Overhead 

First 

First 

3 response 

Learner-centered 

Computer 

student 

student 

4 instruction 


Simulation 

Same 

Same 

5 social comment 

1 ICI IL - 

Demonstrati on 

student 

student 

6activity-related 

L-CI ILCI CU 

Video 

Small 

Small 

comment 

Community- 

Response 

group 

group 

7acknowledge/ praise 

centered 

system 

Large 

Large 

8 guide 

Class 

None 

Group 

Media 

Visitor 

Group 

Media 

Visitor 

9 correction 

0 no response 

A active monitoring 

P passive Monitoring 

Organization 



Figure 1 . VaNTH Observation System Classroom Interaction Observation (CIO) codes (Harris & 


Cox, 2003). 


The HPL Index was created for a couple of reasons. First, unlike previous systems that only 
analyzed CIO data using the how category, the HPL Index groups data into code strings that 
incorporate the who - to whom - what - how - media categories. This allows for a single 
index from which to compare pedagogical styles in traditional and nontraditional engineering 
classrooms and represents the interplay of HPL lenses and other HPL-oriented behaviors 
such as higher order questioning, guidance by an instructor, and group work. Second, each 
of the code strings analyzed using the Index represent traditional instruction, HPL-oriented 
instruction, or organization and sum to 100% of observed classroom time. Only codes 
representing traditional or HPL-oriented instruction, however, are of interest within this 
study. 

Suppose that a professor asks students who are working in groups a higher order question 
about a diagram displayed on the overhead. Within the HPL Index, the corresponding CIO 
code string would be "P-g-2-K/L/A-O" such that "P" represents the professor who is 
initiating the question (who), "g" represents the small group of students to whom the 
professor is asking the question (to whom), and "2" represents the higher order question 
that was asked (what). HPL dimensions represented are knowledge-centered (K), learner- 
centered (L), and assessment-centered (A) since the groups of students are engaging in 
content deeply and the groups are being asked a question about the extent to which they 
understand course material (how). The use of the overhead is represented by "O" (media). 
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Since multiple HPL dimensions are being used, this code string would be categorized within 
the HPL Index as a HPL-oriented instructional code string within an HPL subcategory called 
"higher order questioning by the instructor". 


Previous Work 

To date, the HPL Index has been used to categorize over 36,000 code strings across the 28 
observed courses. HPL-oriented subcategories were categorized based upon their 
integration of HPL-oriented principles, and traditional subcategories were categorized based 
upon their absence of multiple HPL-oriented dimensions. The sum of all HPL-oriented 
instructional subcategories and the sum of all Traditional instructional subcategories were 
found to be statistically significant for both HPL-oriented and traditional courses, thereby 
supporting running additional independent t-tests on each of the 18 subcategories. Using 
independent sample t-tests and a Bonferroni correction to calculate an individual p < 0.003 
for each subcategory, five HPL subcategory items and four traditional subcategory items 
were found to be statistically significant (Table 1). The criterion contrast of the Index was 
confirmed further, since, compared to traditional, lecture-based classes, HPL-designated 
classes reported higher occurrences for all of the statistically significant HPL subcategories, 
and compared to HPL-designated classes, lecture-based classes reported higher occurrences 
for three of the four statistically significant traditional subcategory items. Definitions of each 
of these items along with their frequencies of occurrence within both HPL-oriented and 
traditional, lecture based courses are found in Table 1. 
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Category 

Statistically Significant 
Subcategory & Definition 

H PL- 
Oriented 
Courses 
(N = 17) 

Traditional 
Courses 
(N = ll) 

p value 

HPL 

Guidance by the Instructor. An 
instructor leads students to correct 
answers that they are trying to solve 
in class. 

232 

(0.99%) 

88 

(0.69%) 

0.002*** 


Comment(s). An instructor or 
students make in-class comments 
about academic content using 
multiple HPL dimensions. 

690 

(2.94%) 

222 

(1.74%) 

0.000*** 


Praise. An instructor praises 
students after they respond to an 
HPL-oriented question or HPL- 
oriented comment. 

249 

(1.06%) 

74 

(0.58%) 

0.000*** 


Monitoring. An instructor observes 
students or walks among them as 
they work on in-class activities that 
represent multiple HPL dimensions. 

544 

(2.32%) 

88 

(0.69%) 

0.000*** 


Question and Response. Lower-level 
(yes or no) questions and responses 
to questions that represent multiple 
HPL dimensions. 

836 

(3.56%) 

210 

(1.65%) 

0.000*** 

Traditional 

Instruction by Media. This 
represents media-led classroom 
instruction (e.g., video) that 
represents only the knowledge- 
centered dimension. 

18 

(<0.1%) 

28 

(0.22%) 

0.002*** 


Lecture. This occurs when an 
instructor lectures using only the 
knowledge-centered dimension or 
the knowledge-centered and 
assessment-centered dimensions. 

11354 

(48.41%) 

6729 

(52.85%) 

0.000*** 


Praise. This occurs when an 
instructor praises students after they 
respond to a lower-level "yes" or 
"no" question or to a question that 
does not represent the integration of 
multiple HPL dimensions. 

340 

(1.45%) 

131 

(1.03%) 

0.000*** 


No response. This relates to 
students not responding to a 
professor's question that uses only 
the knowledge-centered dimension 
or knowledge-centered and 
assessment-centered dimensions. 

82 

(0.35%) 

79 

(0.62%) 

0.001*** 


Table 1. Statistically significant HPL-oriented instruction and traditional instruction 


subcategories along with general definitions of the subcategories. 
Family p < 0.05 Corrected individual p < 0.003 


Procedures 

In an effort to understand more about the classroom activities that are associated with the 
frequencies in Table 1 and the corresponding codes strings in Table 2, the objective of the 
current study is to operationalize both HPL-oriented and traditional pedagogical practices 
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associated with the subcategories within this table. This will occur by identifying the 
Classroom Interaction Observation code strings that are represented within each of the 
statistically significant subcategories and by translating them into language representing 
instructional practices that might occur within a classroom setting. In this way, both 
researchers and practitioners can begin to have conversations about practices that are and 
are not occurring within their courses. Only code strings with frequencies greater than 0.1% 
for each of the subcategories will be included in the representations for each subcategory. 

How People Learn-oriented categories 

The five HPL subcategories in which HPL-oriented courses differed from traditional, lecture- 
based courses included the following: (1) guidance by the instructor, (2) question and 
response, (3) monitoring, (4) praise, and (5) comments. The percentage of observed 
instances of occurrence of code strings (across all twenty-eight classes) for the most 
frequently occurring activities within each category are listed in Table 3 followed by a 
discussion of these subcategories and their operationalization. 
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Index 

Subcategory 

CIO Who 
Category 

CIO To 
Whom 
Category 

CIO 

What 

Category 

CIO How 
Category 

CIO Media 
Category 

Percent of 

Observed 

Instances 

HPL 

Guidance by 

P or V 

E 

8 

Any code 
excluding 0 

Any code 
excluding R 

215 

(0.59%) 

the 

professor 

P or V 

S 

8 

Any code 
excluding 0 

Any code 
excluding R 

85 

(0.24%) 

HPL 

Question 

P or V 

g 

1 or 3 

Any code 
excluding 0 

Any code 
excluding R 

59 

(0.16%) 

and 

Response 

P or V 

E 

1 or 3 

KAL, KAC, or 
KLAC 

Any code 
excluding R 

150 

(0.42%) 


P or V 

S 

1 or 3 

KAL, KAC, or 
KLAC 

Any code 
excluding R 

110 

(0.31%) 


E 

P or V 

1 or 3 

KAL, KAC, or 
KLAC 

Any code 
excluding R 

52 

(0.14%) 


g or G 

P 

1 or 3 

Any code 
excluding 0 

Any code 
excluding R 

325 

(0.90%) 


F 

P or V 

1 or 3 

KAL, KAC, or 
KLAC 

Any code 
excluding R 

253 

(0.70%) 


S 

P or V 

1 or 3 

KAL, KAC, or 
KLAC 

Any code 
excluding R 

75 

(0.21%) 

HPL 

Monitoring 

P or V 

g or G 

A or P 

Any code 
excluding 0 

Any code 
excluding R 

410 

(1.14%) 


P or V 

E or F 

A or P 

KAL, KAC, 

KLAC 

Any code 
excluding R 

222 

(0.62%) 

HPL Praise 

P or V 

g 

7 

Any code 
excluding 0 

Any code 
excluding R 

122 

(0.34%) 


P or V 

S 

7 

KAL, KAC, 

KLAC 

Any code 
excluding R 

153 

(0.43%) 

HPL 

Comments 

P 

g 

6 

Any code 
excluding 0 

Any code 
excluding R 

114 

(0.32%) 


g 

p 

6 

Any code 
excluding 0 

Any code 
excluding R 

60 

(0.17%) 


S 

E 

6 

Any code 
excluding 0 

Any code 
excluding R 

174 

(0.48%) 


P or V 

E 

6 

KAL, KAC, 

KLAC 

Any code 
excluding R 

48 

(0.13%) 


P or V 

S 

6 

KAL, KAC, 

KLAC 

Any code 
excluding R 

47 

(0.13%) 


P or V 

E 

6 

KL, KC, or KLC 

Any code 
excluding R 

382 

(1.05%) 


Table 2. Classroom Interaction Observation code strings represented >0.1% of observed instruction 
for statistically significant HPL subcategories. 


Within the sample of bioengineering courses, instructors within HPL courses were more likely 
to guide students to answers instead of just correcting them when they responded 
incorrectly to an answer. More specifically, instructors guided either an entire class to a 
correct answer or guided a single student who had begun previously to talk about a concept 
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within the course. To guide the entire class, the instructor often gave students prompts 
after no one responded to a question. When guiding a single student, the instructor helped 
to clarify or to refine a student's thinking about an academic concept. HPL questions and 
responses occurred when students or instructors asked lower-level (yes or no) questions 
that integrated more than one of the four HPL lenses or when students or instructors 
responded to any question (i.e., higher-order or lower-level) incorporating these lenses. 
Other instances of question and response included an instructor or members of a group 
asking or responding to academic content during small or large group student activities. 
Monitoring involved the instructor engaging with students during group work or the 
instructor watching students as they worked in groups or individually during class. HPL 
praise most likely occurred when the instructor commended a small group as they worked in 
class or when a single student who had begun previously to talk about a concept correctly 
described a concept that represented multiple HPL lenses. Finally, HPL comments most 
often involved an instructor, a small group, or a single student who had begun previously to 
talk about a concept that was not lecture but somehow supplemented the ideas presented 
within the lecture. In addition, instructors were most likely to talk about a tangential 
comment to the entire class or to a student who had begun previously to talk about a 
concept that was a supplement to the lecture content. 

Traditional Instructional Categories 

The four statistically significant traditional subcategories included the following: (1) 
instruction by media, (2) traditional lecture, (3) traditional praise, and (4) no response in a 
traditional environment. The occurrence of these traditional subcategories was more 
prevalent in traditional, lecture-based classes for all of the subcategories except for the 
subcategory of traditional praise. Each of these subcategories and their operationalization 
are discussed below, and the percentage of observed instances of occurrence of code strings 
for the most frequently occurring activities within each category are listed in Table 3. 


Index 

Subcategory 

CIO Who 
Category 

CIO To 
Whom 
Category 

CIO 

What 

Category 

CIO How 
Category 

CIO Media 
Category 

Percent of 

Observed 

Instances 

Instruction by 
Media 

(Traditional) 

M 

E 

4 or 6 

Any code 
excluding 0 

Any code 
excluding R 

46 

(0.13%) 

Traditional 

Lecture 

P or V 

E 

4 

K or KA 

Any code 
excluding R 

18033 

(49.65%) 

Traditional 

Praise 

P or V 

S 

7 

K or KA 

Any code 
excluding R 

433 

(1.20%) 

Traditional, 

No Response 

E 

P or V 

0 

KA 

Any code 
excluding R 

148 

(0.41%) 


Table 3. Classroom Interaction Observation code strings representing >0.1% of observed instruction 
for statistically significant traditional subcategories. 


Within the sample of bioengineering courses, instructors within traditional courses were more 
likely to use instruction by media. In this way, instructors used any form of media to lecture 
to students about course content or to provide supplemental information to students about 
academic content. Traditional lecture was the more commonly used instructional practice 
across the traditional and HPL-oriented courses and was most prevalent when the instructor 
taught academic content to the entire class. Traditional praise, which occurred more often in 
HPL-oriented classes, occurred when an instructor commended a student after 
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he/she provided a correct response to a question asked by the instructor. Finally, no 
response in a traditional environment was prevalent when all students did not respond to 
the instructor's questions about some content that did not represent multiple HPL 
dimensions. 


Discussion 

Within this paper, five HPL-oriented subcategories and four traditional subcategories were 
most likely to occur within observed bioengineering courses. Occurrences within both types 
of courses align with findings observed within innovative courses and in traditional, lecture- 
based courses. For example, the HPL-oriented courses reported effective pedagogy such 
that instructors probed students' understanding of course content by providing reflexive 
tosses (Zee & Minstrell, 1997), by engaging students in higher order questioning and active 
learning via the use of group activities (Springer, et al., 1998), and by involving students in 
inductive teaching and learning approaches and conversations that allow them to think 
about course concepts beyond the traditional academic context (Prince & Felder, 2006). 
Aligned with Boyer's (1990) definition of the scholarship of teaching, these activities 
ultimately can build bridges between teacher's understanding and students' learning and 
allow faculty to be learners as well. On the other hand, traditional activities within observed 
courses paralleled activities that are most likely to occur within lecture-based classroom 
environments. Among these occurrences included uses of technology such that students do 
not engage interactively with the technology and delivery of lecture such that the instructor 
statically stands before students and presents course information without much interaction 
with the students (Felder, Woods, Stice, & Rugarcia, 2000). 

Despite the alignment of the research findings to the findings within previous literature, 
limitations still exist. First, this paper presents the frequencies of occurrences within the 
classrooms, not the sequence of these activities. In other words, although data are reported 
about the kinds of occurrences that occurred most often within undergraduate 
bioengineering courses, no information was presented about the activities that preceded or 
followed these occurrences. Such information could be helpful in developing greater 
understanding about what should be taught within classrooms and when it should be 
taught. Second, data were collected within one discipline (i.e., bioengineering) at one 
university. To understand if the findings are generalizable, additional classroom 
observations within other disciplines and at other universities need to be conducted. Finally, 
since over 100 possible code strings combinations exist and over 30,000 code strings were 
collected, subcategory frequencies, although statistically significant in sum, were relatively 
small when broken down into smaller categories. Because of this, the primary focus of the 
paper was on the operationalization of these code strings, not the magnitude of their 
occurrence within the HPL Index. 

The findings are important for several reasons. It provides insight into the pedagogical 
practices that occur within postsecondary engineering. Although there is literature that 
explores pedagogical practices within engineering (Felder & Silverman, 1988), additional 
information about these practices are needed so that pedagogical practices can be linked 
eventually to student outcomes (e.g., achievement and retention) within observed courses. 
Also, findings reported within the HPL Index raise additional questions about the appropriate 
amount of innovative instruction that is needed within courses. As seen within Table 3, 
although courses are designed to be innovative, the dominant pedagogical practice is still 
lecture. By having a tool such as the Index and by operationalizing the code strings, 
researchers and practitioners can begin to have discussions about the appropriate amount 
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and the types of innovative pedagogy needed to achieve outcomes within courses across a 
variety of contexts. Because the criterion contrast of the HPL Index has been confirmed, 
additional direct observation instruments that explore additional constructs based upon 
pedagogical theories and practices of interest might be developed. 


Conclusions 

This paper details the pedagogical practices that were present across twenty-eight 
(seventeen How People Learn-oriented and eleven traditional, lecture-based) bioengineering 
courses. Although the majority of the occurrences within both types of courses were 
comprised of lecture, other instances of innovative pedagogy occurred predominately within 
courses that were purposefully designed to represent principles of the How People Learn 
framework. The operationalization of both HPL-oriented and traditional subcategories using 
the HPL Index facilitates conversations about ways to connect pedagogical theory to 
practice, provides a snapshot of pedagogy within postsecondary engineering education, and 
lays a foundation for future studies that explore theory-based pedagogical practices in 
multiple environments. 
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